Papers with vocabulary expansion
CORE: Measuring Multi-Agent LLM Interaction Quality under Game-Theoretic Pressures (2026.eacl-long)
Copied to clipboard
| Challenge: | Game-theoretic interactions between agents with large language models (LLMs) have revealed many emergent capabilities, yet the linguistic diversity of these interactions has not been quantified. |
| Approach: | They propose a metric to quantify the effectiveness of language use within multi-agent systems across different game-theoretic interactions. |
| Outcome: | The proposed metric measures the effectiveness of language use within multi-agent systems across game-theoretic interactions. |
Gold Panning in Vocabulary: An Adaptive Method for Vocabulary Expansion of Domain-Specific LLMs (2024.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) struggle when it comes to specialized domains due to limited domain-specific knowledge. |
| Approach: | They propose an adaptive method that automatically identifies valuable words from a given domain vocabulary. |
| Outcome: | The proposed method has been validated on three Chinese datasets and performed on general tasks. |
Language Adaptation of Large Language Models: An Empirical Study on LLaMA2 (2025.coling-main)
Copied to clipboard
| Challenge: | Popularity of Large Language Models (LLMs) has seen a skyrocketing increase in recent years. |
| Approach: | They present a systematic review of the language adaptation process for Large Language Models including vocabulary expansion, continued pre-training, and instruction fine-tuning. |
| Outcome: | The proposed model is based on empirical studies conducted on LLaMA2 and discussions on various settings affecting the model's capabilities. |
Multi-Agent Language Learning: Symbolic Mapping (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on the emergence of language in cooperative tasks where neural network agents learn a communication protocol from scratch to solve problems together. |
| Approach: | They propose a task transfer method and symbolic mapping architecture to help agents learn a compositional and symmetric language in dialog games. |
| Outcome: | The proposed method can help agents learn a compositional and symmetric language in complex settings like dialog games and the proposed architecture promotes vocabulary expansion. |
VEEF-Multi-LLM: Effective Vocabulary Expansion and Parameter Efficient Finetuning Towards Multilingual Large Language Models (2025.coling-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) have a significant disadvantage for low-resource languages . VEEF-Multi-LLM-8B excels in multilingual instruction-following tasks . |
| Approach: | They propose a low-resource multilingual large language model that expands the vocabulary for multilingual support. |
| Outcome: | The proposed model outperforms existing models in multilingual instruction-following tasks, but lags behind English-centric models in some tasks. |
LLaMAX: Scaling Linguistic Horizons of LLM by Enhancing Translation Capabilities Beyond 100 Languages (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) exhibit exceptional translation capabilities in high-resource language tasks, yet their effectiveness in low-resourced languages is suboptimal. |
| Approach: | They conduct extensive multilingual continual pre-training on the LLaMA series models and develop LLiMAX for translation support across more than 100 languages. |
| Outcome: | The proposed model achieves higher translation performance than existing open-source models and performs on-par with specialized translation model on the Flores-101 benchmark. |
Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers (2026.findings-acl)
Copied to clipboard
Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan, Naoto Yokoya
| Challenge: | a new study examines the performance of code-switching IR in monolingual contexts . code-witching is a pervasive linguistic phenomenon in global communication . |
| Approach: | They propose a benchmark to evaluate code-switching IR in monolingual contexts . they propose CS-MTEB, which measures performance declines of up to 27% . |
| Outcome: | The proposed benchmark shows that code-switching performance is degraded by 27% . the proposed benchmark is based on a dataset of mixed-language queries . |
Enhancing Cross-Lingual Transfer through Reversible Transliteration: A Huffman-Based Approach for Low-Resource Languages (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models demonstrate cross-lingual transfer capabilities, but these capabilities often fail to extend to low-resource languages, especially those utilizing non-Latin scripts. |
| Approach: | They propose to combine character transliteration with Huffman coding to create a complete transliterations framework that can be extended to other low-resource languages. |
| Outcome: | The proposed framework reduces storage requirements and improves accuracy and accuracy across multiple downstream tasks while maintaining performance on high-resource languages. |
Multilingual Pixel Representations for Translation and Effective Cross-lingual Transfer (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that pixel representations can be finetuned across scripts without vocabulary extensions, adapters, or transliteration. |
| Approach: | They propose to use pixel representations to train multilingual machine translation models . they explore parameter sharing within and across scripts to better understand where they lead to positive transfer . |
| Outcome: | The proposed model improves on two multilingual datasets with different language coverage compared to subword embeddings . the proposed model can be finetuned cross-lingually or to unseen scripts, and is more data-efficient than other alternatives such as vocabulary expansion . |
VE-KD: Vocabulary-Expansion Knowledge-Distillation for Training Smaller Domain-Specific Language Models (2024.findings-emnlp)
Copied to clipboard
| Challenge: | VE-KD is a method that balances knowledge distillation and vocabulary expansion with the aim of training efficient domain-specific language models. |
| Approach: | They propose a method that balances knowledge distillation and vocabulary expansion with the aim of training efficient domain-specific language models. |
| Outcome: | VE-KD outperforms DistilBERT and Adapt-and-Distill in biomedical domain tasks . compared with other methods, it outperformed Distilbert and adapted-and distill . |
Unleashing the Native Recommendation Potential: LLM-Based Generative Recommendation via Structured Term Identifiers (2026.findings-acl)
Copied to clipboard
Zhiyang Zhang, Junda She, Kuo Cai, Bo Chen, Shiyao Wang, Xinchen Luo, Qiang Luo, Ruiming Tang, Han Li, Kun Gai, Guorui Zhou
| Challenge: | Existing methods for constructing item identifiers face bottlenecks due to their large output space and expensive vocabulary expansion and alignment training. |
| Approach: | They propose to use Large Language Models to develop general-purpose, semantically-aware recommender systems that can be generalized and reusable. |
| Outcome: | Experiments on real-world datasets show that GRAM outperforms baselines and significantly outperformed baselines. |
Mitigating Structural Knowledge Collapse in Domain-Specific LLMs via Morpheme-Aware KV-Aggregation (2026.acl-long)
Copied to clipboard
| Challenge: | Existing tokenizers over-fragment domain terms, disrupting morpheme semantics. |
| Approach: | They propose a lightweight tokenizer that dynamically consolidates fragments without tokenizer changes. |
| Outcome: | The proposed adapter outperforms vocabulary adaptation baselines on medical and legal terms by 3.2–4.6% and 7.9% on high-fragmentation terms. |
Scaling Laws or Threshold Effects: Exploring the Optimal Vocabulary Size for Balancing Performance and Efficiency in Low-Resource Languages (2026.findings-acl)
Copied to clipboard
| Challenge: | vocab expansion scaling laws are well-established for high-resource languages, but they remain unverified in low-resourced settings. |
| Approach: | They propose to scale trilingual vocabulary for languages with 140 to 195,000 tokens . they find that BBPE follows a "decline-then-rise" pattern, whereas BPE improves monotonically . |
| Outcome: | The proposed configuration reduces pre-training duration by over 71% across 1.5B to 8B models while improving downstream performance. |